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About the research 



The challenge of measurement: statistics for planning human resource 
development 

Tom Karmel 

This presentation was made to a breakout session at the 201 1 Skills Australia and Industry Skills 
Councils joint conference, Putting skills at the heart of economy. The paper addresses the challenge 
of measurement in workforce development planning and discusses the role of the various players in 
the labour and training markets - individuals, employers, providers, regulators, governments and 
industry bodies - and the information they need to make optimal decisions. One issue is the extent 
to which markets should be left to operate freely, noting that the data required by central planners 
to second-guess the market are particularly onerous. It is emphasised, however, that markets need 
ample information to function effectively. 

The presentation’s overall assessment is that data in Australia are not too bad, although the lack of 
a complete collection (covering both private and public) of vocational education and training (VET) 
activity is an obvious deficiency. There is also lack of data about the VET workforce (which regulators 
have a particular interest in) and data on job vacancies are limited. Provider-level performance data 
have also not been published to date. 

Tom Karmel 

Managing Director, NCVER 
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The challenge of measurement 




Statistics for planning human 
resource development 



The way I am going to approach the topic is first of all to discuss the concept and then look at the 
roles of the various players involved in some way in human resource development. I will then talk 
about the statistics each player might need and assess the current range of statistics against this list. 
My analysis no doubt reflects my neoclassical economics training and a general scepticism of the value 
of planning for human resource development (manpower planning in another guise?). 

The players 

I’m not totally sure that I understand what human resource development planning actually means, 
but I am interpreting it as the process of employers ensuring that their employees have the necessary 
skills to undertake their jobs. If this is accepted as the definition, I am still unsure how to approach 
the topic, because talking about ‘employees having the necessary skills to undertake their jobs’ 
reflects a very static view of the world. In reality, employers have many choices which impact on 
skills matching. First, they choose what products and services they wish to produce. They then have 
a range of technologies available - which typically trade off different types of labour and labour for 
capital equipment - and within a specific technology there are usually different ways of employing 
labour and capital equipment. Skills can also be learned on the job. Thus employers are faced with 
complex decisions related to ‘employees having the necessary skills’. For example, if labour is short 
or expensive, employers will be more likely to invest in equipment which downplays the importance 
of labour. 

Flowever, putting aside my reservations about the difficulty of nailing the workforce development 
concept, we are essentially talking about two markets: the labour market in which employers and 
individuals trade work for wages; and the training market in which individuals obtain training (and 
therefore acquire skills) from training providers. This characterisation indicates three sets of relevant 
players: employers, individuals, and providers. In addition, one might add the regulators: the relevant 
industrial bodies that regulate the labour market and the training regulators, which regulate the 
quality of the training (noting that education and training is the sort of product which is very difficult 
to assess before it is purchased). 

Thus, if we were to believe that markets operated efficiently, then we would let employers, 
individuals and providers get on with it and not worry too much about planning any overarching 
workforce development. Instead, we would concentrate on information flows to ensure that 
individuals understand the quality of the training they are purchasing and the expected pay-off 
from their investment in skills, and that employers are in a position to judge the skills of potential 
employees. In the case where employers are training their own employees or paying for their training, 
then the information requirements are probably a little less onerous. Thus, under this view of the 
world, wages and conditions, training fees and the like would adjust until we had equilibriums in both 
the labour market and the training market. To promote these adjustments, we would want data on 
the costs and quality of training and the likely employment payoff (that is, the probability of getting 
a job and the wages for particular skills) relating to particular sets of skills. We would also want, 
ideally, information that would enable individuals to understand the future payoff from education, 
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particularly when training for occupations with very long lead times. (Nfedical specialists are an 
obvious example.) Employers may need additional information, such as their rate of labour turnover, 
and an understanding of the labour market in which they operate. 

Table 1 makes an attempt to look at the information needs for each player operating in the labour 
and training market. I do not pretend that is comprehensive, but it does give an idea of the range of 
information that is required to make the labour markets operate efficiently. 

Table 1 I nformation needs to assist labour and training markets 



The player 


Broad area 


Possible indicator 


Assessment of information needs 


Students/employees 


Cost of training 


Fees 


Available for TAFEs but no central source of fees 
for private providers 






Government support 


Numerous websites 




Availability of 
jobs 


Employment growth 


Occupational employment growth from ABS 
labour force survey 






Job vacancies 


ABS survey is high-level. 

Skilled vacancy index from DEEWR (high-level) 
DEEWR research on skills in demand 






Unemployment rates 


Occupational unemployment rates (and be 
qualification level) 






Transitions 


NCVER Student Outcomes Survey, Graduate 
Destination survey 




Quality of training 


Provider indicators 


Not currently produced for the VET sector 




Quality of job 


Extent of permanent 
employment 


ABS Labour Force survey 






Wages 


Occupational wages from ABS 






Stability of employment 


Broad data from ABS Labour Mobility survey 




Future job 
prospects 


Likely growth in job 
opportunities and wages 


Skills Australia’s reports. Skills councils’ 
environmental scans, newspapers, parents 


Employers 


Availability of 
labour 


Numbers of applications 
for jobs 


Internal records 




Knowledge of 
labour market 


Going wage rate 


Occupational wages from ABS plus industry 
knowledge 






Industry labour market 
dynamics 


ABS mobility survey plus industry knowledge 




Labour turnover 




Internal records 


Providers 


Student demand 


Applications for courses 


High-level data from ABS Survey of Education 
and Work; very poor national micro data 




Student 

outcomes 


Proportion in 
employment 


Student Outcomes Survey at state level and or 
large providers (publicly funded only); similar data 
for universities 




Student 

satisfaction 


Level of student 
satisfaction 


Student Outcomes Survey at state level and for 
large providers (publicly funded only) 


VET regulators 


Provider activity 


Knowledge of who is 
providing what 


Only for publicly funded sector 




Completion rates 




Only for publicly funded sector 




Student 

satisfaction 




Only for publicly funded sector 




Quality of staff 




No national collection for VET sector 




Quality of 
assessment 




No independent verification or moderation 
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From table 1 it can be seen that the labour and training markets require copious data if they are to 
function efficiently (and I am not confident I have listed all relevant variables). To the extent that the 
data are not available, individual players are at risk of making sub-optimal decisions. 

I now wish to contrast these data needs with what would be needed if we decide that the market 
should to be managed by regulating the training market; that is, some central planner decides how 
many people should be trained in each course. Essentially, the planner has to internalise or make 
explicit all the knowledge embedded in the decisions of individual market participants. While the 
planner does not need to know everything that an individual employer needs to know (such as internal 
job turnover), he or she does need to model the complicated dynamics of individual labour markets. 
Table 2 gives the flavour of the additional information needs. 

Table 2 Additional data needs for a central planner 



Broad area 


Possible indicator 


Assessment of information needs 


Dynamics of labour market 


Flows in and out of 
occupational labour 
markets 


ABS Mobility survey 


Potential supply 


Number of qualified people 
in workforce 


ABS Labour Force survey 




Wage elasticities (how are 
people attracted back to an 
occupation) 


Good luck 




Potential workforce (how to 
attract non-qualified people 
to occupation) 


Good luck 


Workforce structure 


Age structure 


ABS Labour Force survey and Census 




Retirement patterns 


ABS Mobility survey, synthetic cohort studies 




Factors which affect 
attrition or retirement 


Good luck 


Future of workforce 


Structural changes 


Some patterns are well known; good luck on other 
changes 




Strength of economy 


Good luck; modelling cyclical events for planning 
purposes is well nigh impossible 



As can be seen from table 2, the task facing a central planner is Herculean. While there is good 
information about labour market dynamics, there is a very large number of unknowns that have to be 
modelled (hence the good luck). 

While it is easy to be very critical of the whole notion of central planning, governments are likely to 
be an active player (that is, unwilling to allow markets to operate unhindered) for numerous reasons. 
First, there is the traditional argument of market failure. The two obvious examples are externalities, 
which mean that there is underinvestment in education and training because not all the benefits are 
shared by individuals and employees (one example is the importance of the education of parents 
because of the impact on their children), and capital market constraints, which make it very difficult 
for individuals to borrow in order to invest in their education and training. To address the first of 
these, we would need to estimate the size of externalities. However, no one has had much success 
in actually estimating their size. The second of these issues has been addressed through income- 
contingent loans (VET fee help is the latest of these) and by heavily subsidising fees, as is done in 
the publicly funded part of VET. This role for government does not actually impose additional data 
requirements, except to the extent of needing accurate data on costs of provision and the bad debts 
associated with income- contingent loans. 
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However, governments have at least three other roles: as an employer; as a funder of public services 
such as health, defence and education; and as the body responsible for macroeconomic settings. In all 
of these roles they have a direct interest in influencing the way the market operates. Take education 
for example. Governments have a large interest as both a large employer and as a funder. Like any 
employer, governments prefer to keep the wages of employees as low as possible. However, as a 
funder (as distinct from the role as employer), any increase in wages for teachers has a very large 
impact on the government budget. Therefore, governments wish to ensure that there are ‘sufficient’ 
teachers at a price they wish to pay. So if there is an imbalance in the teacher market, governments 
typically are unwilling to let the market sort it out and would prefer to increase the supply of 
teachers in order to keep wages down. In this context, manpower planning is an attractive idea - by 
understanding the flows of labour governments can try to anticipate how many teachers they should 
train in order to keep the system working without any significant wage breakouts. (This constraint can 
have some undesirable side effects which are realised changes to the quality of teachers.) Health is 
another example, and in fact there is a very extensive bureaucracy undertaking health and medical 
workforce planning. 

The role of government as a funder has important implications for data collection. While individual 
employers need to understand the dynamics of their own firm, governments need to understand the 
dynamics of the industries where they are large funders. They need accurate workforce data, an 
understanding of attrition rates and so on. This applies particularly to health, education, community 
services and defence. As an aside, changes in government regulatory policies can have a major impact 
on the labour markets of industries. For example, making qualifications mandatory in say, child care, 
will have very significant flow-on effects to the child care market and the child carers’ market. 

The role of governments as the custodian of the macroeconomy is a little different. Even if 
governments had no employees in industries such as health and education, they still have an interest 
in what is going on labour markets. That interest centres around two areas. First, there is always 
a focus on inflation and unemployment. If general wage inflation increases, then the Reserve Bank 
(which for convenience I am including as part of government) may well increase interest rates to slow 
the economy down. The second area governments are particularly interested in is that of structural 
imbalances. Thus inadequate infrastructure (roads, ports etc.) will limit the growth rate of the 
economy. Similarly, constraints in the availability of skilled labour may limit the growth rate of the 
economy. This is what drives the government’s interest in the skill requirements of the mining and 
resources industry. If we are worried about avoiding structural imbalances, then Richardson (2007) 
and Skills Australia (2011) argue that we only need to be concerned with occupations for which 
training is very lengthy or are critical for the broader economy. In table 3, I list the major information 
needs of governments worried about cyclical or structural developments. The variables revolve around 
common economic indicators and are generally pretty well covered by the ABS Labour Force survey. 

To date, I have not discussed the role of ‘industry’. I have always had some difficulty understanding 
the role of ‘industry’ because industry bodies are made up of individual employers, all of whom have 
rather different interests (and competition policy generally prohibits collusion). However, industry 
has played an important role in relation to education and training in two main ways. The first is in 
influencing the actual training through the content of training packages. The second is in combining 
to share training facilities - there are good examples in the construction industry but they occur in a 
range of industries. Group training companies are another example of an industry body which acts as 
an intermediary between firms and individuals and which makes it easier for small and medium-size 
firms to take on apprentices or trainees. 
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Table 3 Labour market data needs for governments as custodians of the economy 



Strength of the labour 


Employment growth 


ABS Labour Force survey 


market 


Unemployment rate 


ABS Labour Force survey 




Job vacancies 


ABS Job Vacancies survey 




Overtime usage 


ABS Job Vacancies and 
Overtime survey (until 1999) 




Wage growth 


ABS Labour Force survey 


Structural mismatch 


Persistent low unemployment rates and high 
wages or wage growth 


ABS Labour Force survey 




Persistent underutilisation of labour 


ABS Labour Force survey 



It is not obvious that industry, in this sense, imposes significant additional data requirements. 

However, as developers of training packages, industry should ensure that development is effective and 
efficient, so data on the uptake of training packages and modules are important. 

I have almost finished but there are a couple of aspects I wish to discuss. The first is industry as 
a lobby group for increased government expenditure. The sharing of costs of training between 
individuals, employers and the government is obviously a contested area. Industry tends to talk 
about skill shortages or skill deficits and lobbies government to spend more training dollars on their 
areas of concern. To the extent that they are successful, this obviates the need for the industry itself 
to spend the money on training. It also keeps wages down. As an employer, I much prefer facing a 
labour market in which I can readily recruit people with the requisite skills at a modest (but fair of 
course) wage. As an employer, I am very happy to make use of government-subsidised training for my 
employees. Of course, industry groups tend to point to ‘market failure’ as the pretext for request 
for government funding. But market failure is a slippery concept and is often used inappropriately to 
refer to a situation which one of the parties does not like; from an economic perspective there is no 
market failure if the benefits are split between employers and employees. 

The final aspect is government as a funder of general education. The discussion above has focused 
on the labour market, and it would be remiss not to refer to the government’s role in funding 
general education. In fact, one could argue that perhaps the best thing that a government can do for 
workforce development is to ensure that all of its citizens have a good general education. In times 
past, this has perhaps been taken to be up to the end of compulsory education. In the modern world, 
one could argue that the government should ensure that all citizens have the opportunity to obtain 
general and entry-level training, perhaps up to an initial degree. 

Final comment 

Data needs do depend on the extent to which central planning is practised. Central planning is 
demanding on the data, but there is also no denying that markets work better when there is prolific 
information. Overall, our data are not too bad, although there are some clear gaps, such as the lack 
of comprehensive data on both public and private VET. There are also poor data on the VET workforce 
(of particular interest to regulators) and job vacancy data are limited at the occupation level. 
Provider- level performance data have also not been published to date. Much of the relevant data are 
provided by the regular Australian Bureau of Statistics and NCVER collections, but it is acknowledged 
that there are clear limits to how far data can be disaggregated. Thus local knowledge is important 
and it is unlikely that official data collections can ever obviate the need for them. 
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It is worth noting that I have been silent on one of the suggestions in the advertising blurb for this 
session; that is, the possibility of running a National Employer Skills Survey to provide more detailed 
information on skill shortages and skill gaps.' The UK National Employer Skills Survey, now called the 
UK Employer Perspectives Survey, has been run for some years to obtain ‘robust evidence regarding 
employers’ engagement and satisfaction with government support for recruitment and workforce 
development’ (UK Commission for Employment and Skills 2011). The purpose of the survey seems to 
presuppose that there should be government support for recruitment and workforce development. 

On this basis, the survey is more a customer satisfaction survey than one necessary for efficient 
workforce development. Australia does have a similar survey which is conducted every two years, the 
Survey of Employer Use and Views of the VET System, which provides high-level data on the use of and 
satisfaction with the Australian VET system. However, my view is that, while this type of survey does 
give an overall feel for the engagement of Australian industry with education and training, it is not a 
substitute for hard data on wages, unemployment rates, vacancy rates and the like. 



1 See Shah and Burke (2003) for a discussion of the various concepts relating to skills shortages; also Department of 
Education, Employment and Workplace Relations (2008, 2009). 



12 



The challenge of measurement 




References 

Department of Education, Employment and Workplace Relations 2008, ‘Skill shortage methodology 2008-09’, 
DEEWR, Canberra. 

— 2009, Skilled vacancy index, November, DEEWR, Canberra. 

Richardson, S 2007, What is a skills shortage? NCVER, Adelaide. 

Shah, C £t Burke, G 2003, Skills shortages: concepts, measurement and implications, Centre for the 
Economics of Education & Training, Melbourne 

Skills Australia 201 1 , Skills for prosperity: a road map for vocational education and training, viewed 7 
August 201 1 , <http://www.skillsaustralia.gov.au/PDFs_RTFs/SkillsProsperityRoadmap.pdf>. 

UK Commission for Employment and Skills 2011, Executive summary 25, UK Employer Perspectives Survey 
2010, HMSO, London. 



NCVER 



13 






NCVER 



National Centre for \bcational Education Research Ltd 

Level 11, 33 King William Street, Adelaide, South Australia 
PO Box 8288, Station Arcade, SA 5000 Australia 
Telephone -+61 8 8230 8400 Facsimile +61 8 8212 3436 
Website www. never edu. au Email ncver@ncver edu. au 




